Gray Bots Surge as Generative AI Scraper Activity Increases

A recent surge in generative AI scraper bot activity has been observed impacting the online landscape. New data indicates that these “gray bots” are increasingly targeting web applications.

Barracuda’s latest report, Generative AI Bot Activity Trends, highlights the growing presence of AI-driven bots that aggressively collect online data.

The Rise of Gray Bots

Between December 2024 and February 2025, millions of requests were received by web applications from generative AI bots such as ClaudeBot and TikTok’s Bytespider.

In a single 30-day period, one tracked web application logged 9.7 million bot requests, while another faced over 500,000 bot requests in just one day. Further analysis found that one web application experienced 17,000 bot requests per hour over 24 hours.

Unlike traditional bots that operate in bursts, these generative AI scraper bots maintain consistent traffic levels. This unexpected pattern creates significant challenges for web applications, making it harder to predict and mitigate their impact.

Gray bots, while not explicitly malicious, can be highly disruptive.

Their aggressive scraping can:

• Overwhelm web application traffic, disrupting normal operations
• Extract and use copyrighted data without authorization
• Distort website analytics, affecting business decision-making
• Increase cloud hosting costs due to higher CPU and bandwidth usage
• Raise compliance risks in industries handling sensitive data, such as healthcare and finance

Read more on the legal implications of AI-driven data scraping: ChatGPT’s Data-Scraping Model Under Scrutiny From Privacy Experts

Two of the most prolific generative AI scraper bots detected in early 2025 are ClaudeBot and Bytespider.

ClaudeBot, operated by Anthropic, collects data to train its generative AI model, Claude. Despite its aggressive scraping, Anthropic provides information on how to block its activity.

Bytespider, TikTok’s AI scraper bot, gathers data to refine its recommendation algorithms and advertising features. Reports indicate that Bytespider operates with little transparency, making it difficult for web applications to manage its impact.

Other notable bots detected include PerplexityBot and DeepSeekBot.

Strategies for Protection

With gray bots becoming a persistent part of online traffic, organizations must take proactive steps to manage their impact. One common approach is deploying robots.txt, a tool that signals scrapers to avoid collecting site data. However, this method is not legally enforceable and many bots ignore it.

For more effective protection, companies are turning to AI-powered bot defense systems that leverage machine learning to detect and block scraper bot activity in real time.

As debates over the ethical, legal and commercial implications of AI scraper bots continue, organizations must prioritize security to safeguard their data and operations.